Systems and methods for contract based offer generation

ABSTRACT

Systems and methods for a contract-based offer generator is provided. A contract for a promotional offer on a product is received. Data is extracted from the contract. An offer band is accessed, and a plurality of test offers are selected from the offer bank by scoring each offer in the offer bank against the extracted data. The promotional offer and the selected plurality of test offers are deployed in a plurality of retail locations. This is done by maximizing orthogonality between the following variables: store sales, store out of stock rates, number of relevant SKUs carried in each store, temporal effects, discount depth, buy quantity and offer structure.

CROSS REFERENCE TO RELATED APPLICATIONS

This non-provisional application claims priority to U.S. ProvisionalApplication No. 63/143,847, filed Jan. 30, 2021, of the same inventorsand title, pending, which application is incorporated herein in itsentirety by this reference.

Additionally, this is a continuation-in-part application which claimsthe benefit of U.S. application entitled “Systems and Methods forIntelligent Promotion Design in Bric and Mortar Retailers with PromotionScoring,” U.S. application Ser. No. 16/120,178, filed Aug. 31, 2018, byRapperport et al., (Attorney Docket No. EVS-1801-C1), pending, which isa continuation application and claims the benefit of U.S. applicationSer. No. 15/990,005, filed May 25, 2018, of the same title, (AttorneyDocket No. EVS-1801), abandoned, which is a continuation-in-partapplication and claims the benefit of U.S. application Ser. No.14/209,851, filed Mar. 13, 2014, entitled “Architecture and Methods forPromotion Optimization,” by Moran (Attorney Docket No. EVS-1401), nowU.S. Pat. No. 9,984,387 issued May 29, 2018, which claims priority under35 U.S.C. 119(e) to a commonly owned U.S. Provisional Application No.61/780,630, filed Mar. 13, 2013, entitled “Architecture and Methods forPromotion Optimization,” by Moran (Attorney Docket No. PRCO-P001P1),expired. application Ser. No. 15/990,005 also claims the benefit of U.S.Provisional Application No. 62/576,742, filed Oct. 25, 2017, entitled“Architecture and Methods for Generating Intelligent Offers with DynamicBase Prices” by Rapperport et al. (Attorney Docket No. EVS-1703-P),expired. Additionally, U.S. application Ser. No. 16/120,178 claimspriority to U.S. Provisional Application No. 62/553,133, filed Sep. 1,2017, entitled “Systems and Methods for Promotion Optimization” byRapperport et al. (Attorney Docket No. EVS-170X-P), expired.

This continuation-in-part application also claims the benefit of U.S.application entitled “Systems and Methods for Price Testing andOptimization in Brick and Mortar Retailers,” U.S. application Ser. No.16/157,018, filed Oct. 10, 2018, by Montero et al., (Attorney Docket No.EVS-1802), now U.S. Pat. No. 10,915,912 issued Feb. 9, 2021, and claimsthe benefit of U.S. application entitled “Systems and Methods forCollaborative Offer Generation,” U.S. application Ser. No. 16/216,997,filed Dec. 11, 2018, by Ventrice et al., (Attorney Docket No. EVS-1803),recently allowed.

The present invention is additionally related to the followingapplications and patents, all of which are incorporated herein byreference:

Commonly owned U.S. application Ser. No. 14/231,426, filed on Mar. 31,2014, entitled “Adaptive Experimentation and Optimization in AutomatedPromotional Testing,” by Moran et al. (Attorney Docket No. EVS-1402),now U.S. Pat. No. 10,438,230 issued Oct. 8, 2019.

Commonly owned U.S. application Ser. No. 14/231,432, filed on Mar. 31,2014, entitled “Automated and Optimal Promotional Experimental TestDesigns Incorporating Constraints,” by Moran et al. (Attorney Docket No.EVS-1403), now U.S. Pat. No. 9,940,639 issued Apr. 10, 2018.

Commonly owned U.S. application Ser. No. 14/231,440, filed on Mar. 31,2014, entitled “Automatic Offer Generation Using Concept GeneratorApparatus and Methods Therefor,” by Moran et al. (Attorney Docket No.EVS-1404)), now U.S. Pat. No. 10,438,231, issued Oct. 8, 2019.

Commonly owned U.S. application Ser. No. 14/231,442, filed on Mar. 31,2014, entitled “Automated Event Correlation to Improve PromotionalTesting,” by Moran et al. (Attorney Docket No. EVS-1405), now U.S. Pat.No. 9,940,640 issued Apr. 10, 2018.

Commonly owned U.S. application Ser. No. 14/231,460, filed on Mar. 31,2014, entitled “Automated Promotion Forecasting and Methods Therefor,”by Moran et al. (Attorney Docket No. EVS-1406)), now U.S. Pat. No.10,445,763, issued Oct. 15, 2019.

Commonly owned U.S. application Ser. No. 14/231,555, filed on Mar. 31,2014, entitled “Automated Behavioral Economics Patterns in PromotionTesting and Methods Therefor,” by Moran et al. (Attorney Docket No.EVS-1407), now U.S. Pat. No. 10,140,629 issued Nov. 27, 2018.

All the applications/patents listed above are incorporated herein intheir entirety by this reference.

BACKGROUND

The present invention relates generally to offer generation methods andapparatus therefor. More particularly, the present invention relates tocomputer-implemented methods and computer-implemented apparatus for thegeneration of promotions leveraging existing contracts a retailer entersinto in order to determine the best promotional variable values.

Promotion refers to various practices designed to increase sales of aparticular product or services and/or the profit associated with suchsales. Generally speaking, the public often associates promotion withthe sale of consumer goods and services, including consumer packagedgoods (e.g., food, home and personal care), consumer durables (e.g.,consumer appliances, consumer electronics, automotive leasing), consumerservices (e.g., retail financial services, health care, insurance, homerepair, beauty and personal care), and travel and hospitality (e.g.,hotels, airline flights, and restaurants). Promotion is particularlyheavily involved in the sale of consumer packaged goods (e.g., consumergoods packaged for sale to an end consumer). However, promotion occursin almost any industry that offers goods or services to a buyer (whetherthe buyer is an end consumer or an intermediate entity between theproducer and the end consumer).

The term promotion may refer to, for example, providing discounts (usingfor example a physical or electronic coupon or code) designed to, forexample, promote the sales volume of a particular product or service.One aspect of promotion may also refer to the bundling of goods orservices to create a more desirable selling unit such that sales volumemay be improved. Another aspect of promotion may also refer to themerchandising design (with respect to looks, weight, design, color,etc.) or displaying of a particular product with a view to increasingits sales volume. It includes calls to action or marketing claims usedin-store, on marketing collaterals, or on the package to drive demand.Promotions may be composed of all or some of the following: price basedclaims, secondary displays or aisle end-caps in a retail store, shelfsignage, temporary packaging, placement in a retailercircular/flyer/coupon book, a colored price tag, advertising claims, orother special incentives intended to drive consideration and purchasebehavior. These examples are meant to be illustrative and not limiting.

In addition to promotional activities, it is also desirable to performoptimizations of base pricing (e.g. non-promotional prices). Oftenretailers rely upon manufacturer's suggested retail pricing (MSRP) forsetting of base prices. In other circumstances, base prices are setbased upon competitive analysis—a retailer may monitor competitor's andmatch or beat the competitor's price on some or all the goods in astore. Alternatively, some retailers may set a desired margin, or salesvolume, for a good, and set prices accordingly. Generally however, thebase prices of goods in a brick-and-mortar store do not varysignificantly due to logistical concerns of updating signage and pointof sales (POS) databases, consumer expectation of generally consistentbase prices, and the tendency that a retailer will continue patterns ofbehavior (e.g., “this is what we have always done”).

In discussing various embodiments of the present invention, the sale ofconsumer packaged goods (hereinafter “CPG”) is employed to facilitatediscussion and ease of understanding. It should be kept in mind,however, that the promotion and base pricing optimization methods andapparatuses discussed herein may apply to any industry in which there isany pricing flexibility in the past or may be employed in the future.

Further, price discount is employed as an example to explain thepromotion methods and apparatuses herein. It should be understood,however, that promotion optimization may be employed to manipulatefactors other than price discount in order to influence the salesvolume. An example of such other factors may include the call to actionon a display or on the packaging, the size of the CPG item, the mannerin which the item is displayed or promoted or advertised either in thestore or in media, etc.

Generally speaking, it has been estimated that, on average, 17% of therevenue in the consumer packaged goods (CPG) industry is spent to fundvarious types of promotions, including discounts, designed to enticeconsumers to try and/or to purchase the packaged goods. In a typicalexample, the retailer (such as a grocery store) may offer a discountonline or via a print circular to consumers. The promotion may bespecifically targeted to an individual consumer (based on, for example,that consumer's demographics or past buying behavior). The discount mayalternatively be broadly offered to the general public. Examples ofpromotions offered to general public include for example, a printed orelectronic redeemable discount (e.g., coupon or code) for a specific CPGitem. Another promotion example may include, for example, generaladvertising of the reduced price of a CPG item in a particulargeographic area. Another promotion example may include in-store markingdown of a particular CPG item only for a loyalty card user base.

In an example, if the consumer redeems the coupon or electronic code,the consumer is entitled to a reduced price for the CPG item. Therevenue loss to the retailer due to the redeemed discount may bereimbursed, wholly or partly, by the manufacturer of the CPG item in aseparate transaction.

Because promotion and base price testing is expensive (in terms of, forexample, the effort to conduct a promotion campaign, modify displayprices and/or the per-unit revenue loss to the retailer/manufacturerwhen the consumer decides to take advantage of the discount), effortsare continually made to minimize promotion cost while maximizing thereturn on promotion dollars investment. This effort is known in theindustry as promotion optimization.

For example, a typical promotion optimization method may involveexamining the sales volume of a particular CPG item over time (e.g.,weeks). The sales volume may be represented by a demand curve as afunction of time, for example. A demand curve lift (excess overbaseline) or dip (below baseline) for a particular time period would beexamined to understand why the sales volume for that CPG item increasesor decreases during such time period.

FIG. 1 shows an example demand curve 102 for Brand X cookies over someperiod of time. Two lifts 110 and 114 and one dip 112 in demand curve102 are shown in the example of FIG. 1. Lift 110 shows that the demandfor Brand X cookies exceeds the baseline at least during week 2. Byexamining the promotion effort that was undertaken at that time (e.g.,in the vicinity of weeks 1-4 or week 2) for Brand X cookies, marketershave in the past attempted to judge the effectiveness of the promotioneffort on the sales volume. If the sales volume is deemed to have beencaused by the promotion effort and delivers certain financialperformance metrics, that promotion effort is deemed to have beensuccessful and may be replicated in the future in an attempt to increasethe sales volume. On the other hand, dip 112 is examined in an attemptto understand why the demand falls off during that time (e.g., weeks 3and 4 in FIG. 1). If the decrease in demand was due to the promotion inweek 2 (also known as consumer pantry loading or retailerforward-buying, depending on whether the sales volume shown reflects thesales to consumers or the sales to retailers), this decrease in weeks 3and 4 should be counted against the effectiveness of week 2.

One problem with the approach employed in the prior art has been thefact that the prior art approach is a backward-looking approach based onaggregate historical data. In other words, the prior art approachattempts to ascertain the nature and extent of the relationship betweenthe promotion and the sales volume by examining aggregate data collectedin the past. The use of historical data, while having some disadvantages(which are discussed later herein below), is not necessarily a problem.However, when such data is in the form of aggregate data (such as insimple terms of sales volume of Brand X cookies versus time for aparticular store or geographic area), it is impossible to extract fromsuch aggregate historical data all of the other factors that may morelogically explain a particular lift or dip in the demand curve.

To elaborate, current promotion and base price optimization approachestend to evaluate sales lifts or dips as a function of four main factors:discount depth (e.g., how much was the discount on the CPG item),discount duration (e.g., how long did the promotion campaign last),timing (e.g., whether there was any special holidays or event or weatherinvolved), and promotion type when analyzing for promotions (e.g.,whether the promotion was a price discount only, whether Brand X cookieswere displayed/not displayed prominently, whether Brand X cookies werefeatures/not featured in the promotion literature).

However, there may exist other factors that contribute to the sales liftor dip, and such factors are often not discoverable by examining, in abackward-looking manner, the historical aggregate sales volume data forBrand X cookies. This is because there is not enough information in theaggregate sales volume data to enable the extraction of informationpertaining to unanticipated or seemingly unrelated events that may havehappened during the sales lifts and dips and may have actuallycontributed to the sales lifts and dips.

Suppose, for example, that there was a discount promotion for Brand Xcookies during the time when lift 110 in the demand curve 102 happens.However, during the same time, there was a breakdown in the distributionchain of Brand Y cookies, a competitor's cookies brand which manyconsumers view to be an equivalent substitute for Brand X cookies. WithBrand Y cookies being in short supply in the store, many consumersbought Brand X instead for convenience sake. Aggregate historical salesvolume data for Brand X cookies, when examined after the fact inisolation by Brand X marketing department thousands of miles away, wouldnot uncover that fact. As a result, Brand X marketers may make themistaken assumption that the costly promotion effort of Brand X cookieswas solely responsible for the sales lift and should be continued,despite the fact that it was an unrelated event that contributed to mostof the lift in the sales volume of Brand X cookies.

As another example, suppose, for example, that milk produced by aparticular unrelated vendor was heavily promoted in the same grocerystore or in a different grocery store nearby during the week that BrandX cookies experienced the sales lift 110. The milk may have beenhighlighted in the weekly circular, placed in a highly visible locationin the store and/or a milk industry expert may have been present in thestore to push buyers to purchase milk, for example. Many consumers endedup buying milk because of this effort whereas some of most of thoseconsumers who bought during the milk promotion may have waited anotherweek or so until they finished consuming the milk they bought in theprevious weeks. Further, many of those milk-buying consumers during thisperiod also purchased cookies out of an ingrained milk-and-cookieshabit. Aggregate historical sales volume data for Brand X cookies wouldnot uncover that fact unless the person analyzing the historicalaggregate sales volume data for Brand X cookies happened to be presentin the store during that week and had the insight to note that milk washeavily promoted that week and also the insight that increased milkbuying may have an influence on the sales volume of Brand X cookies.

Software may try to take some of these unanticipated events into accountbut unless every SKU (stock keeping unit) in that store and in storeswithin commuting distance and all events, whether seemingly related orunrelated to the sales of Brand X cookies, are modeled, it is impossibleto eliminate data noise from the backward-looking analysis based onaggregate historical sales data.

Even without the presence of unanticipated factors, a marketing personworking for Brand X may be interested in knowing whether the relativelymodest sales lift 114 comes from purchases made by regular Brand Xcookies buyers or by new buyers being enticed by some aspect of thepromotion campaign to buy Brand X cookies for the first time. If Brand Xmarketer can ascertain that most of the lift in sales during thepromotion period that spans lift 114 comes from new consumers of Brand Xcookies, such marketer may be willing to spend more money on the sametype of sales promotion, even to the point of tolerating a negative ROI(return on investment) on his promotion dollars for this particular typeof promotion since the recruitment of new buyers to a brand is deemedmore much valuable to the company in the long run than the temporaryincrease in sales to existing Brand X buyers. Again, aggregatehistorical sales volume data for Brand X cookies, when examined in abackward-looking manner, would not provide such information.

Furthermore, even if all unrelated and related events and factors can bemodeled, the fact that the approach is backward-looking means that thereis no way to validate the hypothesis about the effect an event has onthe sales volume since the event has already occurred in the past. Withrespect to the example involving the effect of milk promotion on Brand Xcookies sales, there is no way to test the theory short of duplicatingthe milk shortage problem again. Even if the milk shortage problem couldbe duplicated again for testing purposes, other conditions have changed,including the fact that most consumers who bought milk during thatperiod would not need to or be in a position to buy milk again in a longtime. Some factors, such as weather, cannot be duplicated, making theoryverification challenging.

Attempts have been made to employ non-aggregate sales data in promotingproducts. For example, some companies may employ a loyalty card program(such as the type commonly used in grocery stores or drug stores) tokeep track of purchases by individual consumers. If an individualconsumer has been buying sugar-free cereal, for example, themanufacturer of a new type of whole grain cereal may wish to offer adiscount to that particular consumer to entice that consumer to try outthe new whole grain cereal based on the theory that people who boughtsugar-free cereal tend to be more health conscious and thus more likelyto purchase whole grain cereal than the general cereal-consuming public.Such individualized discount may take the form of, for example, aredeemable discount such as a coupon or a discount code mailed oremailed to that individual.

Some companies may vary the approach by, for example, ascertaining theitems purchased by the consumer at the point of sale terminal andoffering a redeemable code on the purchase receipt. Irrespective of theapproach taken, the utilization of non-aggregate sales data hastypically resulted in individualized offers, and has not been processedor integrated in any meaningful sense into a promotion optimizationeffort to determine the most cost-efficient, highest-return manner topromote a particular CPG item to the general public.

Attempts have also been made to obtain from the consumers themselvesindications of future buying behavior instead of relying on abackward-looking approach. For example, conjoint studies, one of thestated preference methods, have been attempted in which consumers areasked to state preferences. In an example conjoint study, a consumer maybe approached at the store and asked a series of questions designed touncover the consumer's future shopping behavior when presented withdifferent promotions. Questions may be asked include, for example, “doyou prefer Brand X or Brand Y” or “do you spend less than $100 or morethan $100 weekly on grocery” or “do you prefer chocolate cookies oroatmeal cookies” or “do you prefer a 50-cent-off coupon or a 2-for-1deal on cookies”. The consumer may state his preference on each of thequestions posed (thus making this study a conjoint study on statedpreference).

However, such conjoint studies have proven to be an expensive way toobtain non-historical data. If the conjoint studies are presented via acomputer, most users may ignore the questions and/or refuse toparticipate. If human field personnel are employed to talk to individualconsumers to conduct the conjoint study, the cost of such studies tendsto be quite high due to salary cost of the human field personnel and maymake the extensive use of such conjoint studies impractical.

Further and more importantly, it has been known that conjoint studiesare somewhat unreliable in gauging actual purchasing behavior byconsumers in the future. An individual may state out of guilt and theknowledge that he needs to lose weight that he will not purchase anycookies in the next six months, irrespective of discounts. In actuality,that individual may pick up a package of cookies every week if suchpackage is carried in a certain small size that is less guilt-inducingand/or if the package of cookies is prominently displayed next to themilk refrigerator and/or if a 10% off discount coupon is available. If apromotion effort is based on such flawed stated preference data,discounts may be inefficiently deployed in the future, costing themanufacturer more money than necessary for the promotion.

Finally, none of the approaches track the long-term impact of apromotion's effect on brand equity for an individual's buying behaviorover time. Some promotions, even if deemed a success by traditionalshort-term measures, could have damaging long-term consequences.Increased price-based discounting, for example, can lead to consumersincreasing the weight of price in determining their purchase decisions,making consumers more deal-prone and reluctant to buy at full price,leading to less loyalty to brands and retail outlets.

It is therefore apparent that an urgent need exists for systems andmethods that enable improvements in the generation of offers on productsthat have been contracted with the retailer.

SUMMARY

To achieve the foregoing and in accordance with the present invention,systems and methods for contract based offer generation is provided. Insome embodiments, a contract for a promotion is received by theretailer. Often these promotion contracts are periodically negotiatedbetween the retailer and a distributor or manufacturer. Transaction logsof a retailer are accessed to determine offer variants, which are storedin an offer bank. These collected offers are then compared against thecontracted offer and scored accordingly to select and administer aselect set of offers. This causes improved sales (or other objective),and provides additional feedback data to further refine the bestpossible promotional activities.

In some embodiments, a contract for a promotional offer on a product isreceived. Data is extracted from the contract. An offer band isaccessed, and a plurality of test offers are selected from the offerbank by scoring each offer in the offer bank against the extracted data.The promotional offer and the selected plurality of test offers aredeployed in a plurality of retail locations. This is done by maximizingorthogonality between the following variables: store sales, store out ofstock rates, number of relevant SKUs carried in each store, temporaleffects, discount depth, buy quantity and offer structure.

The selecting the number of test offers to run in-market is done usingreinforcement learning techniques, and in particular Thompson sampling.The offer bank is populated with forecasted offers, which are based upontransaction logs of a plurality of retailers. The transaction logs areadjusted for compliance by the given retailer, estimated out of stockevents, normalized across stores to account for different storeattributes, and adjusted for temporal effects. Machine learning isapplied to the adjusted transaction logs to determine lift and standarddeviation for a given test offer. The forecasts are a baseline functionof time from the transaction log data plus elasticity from cross storeexperiments times a change in price, where in the elasticity iscalculated as a function of the lift, and a confidence for the forecastis calculated as a function of the standard deviation.

Note that the various features of the present invention described abovemay be practiced alone or in combination. These and other features ofthe present invention will be described in more detail below in thedetailed description of the invention and in conjunction with thefollowing figures.

BRIEF DESCRIPTION OF THE DRAWINGS

In order that the present invention may be more clearly ascertained,some embodiments will now be described, by way of example, withreference to the accompanying drawings, in which:

FIG. 1 shows an example demand curve 102 for Brand X cookies over someperiod of time;

FIG. 2A shows, in accordance with an embodiment of the invention, aconceptual drawing of the forward-looking promotion optimization method;

FIG. 2B shows, in accordance with an embodiment of the invention, thesteps for generating a general public promotion;

FIG. 3A shows in greater detail, in accordance with an embodiment of theinvention, the administering step 206 of FIG. 2 from the user'sperspective;

FIG. 3B shows in greater detail, in accordance with an embodiment of theinvention, the administering step 206 of FIG. 2 from the forward-lookingpromotion optimization system perspective;

FIG. 4 shows various example segmentation criteria that may be employedto generate the purposefully segmented subpopulations;

FIG. 5 shows various example methods for communicating the testpromotions to individuals of the segmented subpopulations being tested;

FIG. 6 shows, in accordance with some embodiments, various examplepromotion-significant responses;

FIG. 7 shows, in accordance with some embodiments, various example testpromotion variables affecting various aspects of a typical testpromotion;

FIG. 8 shows, in accordance with some embodiments, a generalhardware/network view of a forward-looking promotion optimizationsystem;

FIG. 9 shows, in accordance with some embodiments, a block diagram of anoffer collaboration architecture;

FIG. 10 shows, in accordance with some embodiments, an exampleillustration of a contract based offer generation system;

FIG. 11 shows, in accordance with some embodiments, a flowchart of anexample method for the generation and administration of a contract basedoffer;

FIG. 12 shows, in accordance with some embodiments, a flowchart of anexample method for contract parsing;

FIG. 13 shows, in accordance with some embodiments, a flowchart of anexample method for offer generation;

FIG. 14 shows, in accordance with some embodiments, an examplescreenshot of a contract based offer management system at the categorylevel;

FIG. 15 shows, in accordance with some embodiments, an examplescreenshot of a contract based offer management system at the productlevel;

FIG. 16 shows, in accordance with some embodiments, an examplescreenshot of a contract based offer management system at a particularlypromotional event;

FIG. 17 shows, in accordance with some embodiments, an examplescreenshot of a contract based offer management system at for theresults of execution of a particular promotion event;

FIG. 18 shows, in accordance with some embodiments, an examplescreenshot of a contract based offer management system for editing apromotional event;

FIG. 19 shows, in accordance with some embodiments, an examplescreenshot of a contract based offer management system illustrating theoffer bank; and

FIGS. 20A and 20B are example computer systems capable of implementingthe systems for offer generation and administration.

DETAILED DESCRIPTION

The present invention will now be described in detail with reference toseveral embodiments thereof as illustrated in the accompanying drawings.In the following description, numerous specific details are set forth inorder to provide a thorough understanding of embodiments of the presentinvention. It will be apparent, however, to one skilled in the art, thatembodiments may be practiced without some or all of these specificdetails. In other instances, well known process steps and/or structureshave not been described in detail in order to not unnecessarily obscurethe present invention. The features and advantages of embodiments may bebetter understood with reference to the drawings and discussions thatfollow.

Aspects, features and advantages of exemplary embodiments of the presentinvention will become better understood with regard to the followingdescription in connection with the accompanying drawing(s). It should beapparent to those skilled in the art that the described embodiments ofthe present invention provided herein are illustrative only and notlimiting, having been presented by way of example only. All featuresdisclosed in this description may be replaced by alternative featuresserving the same or similar purpose, unless expressly stated otherwise.Therefore, numerous other embodiments of the modifications thereof arecontemplated as falling within the scope of the present invention asdefined herein and equivalents thereto. Hence, use of absolute and/orsequential terms, such as, for example, “will,” “will not,” “shall,”“shall not,” “must,” “must not,” “first,” “initially,” “next,”“subsequently,” “before,” “after,” “lastly,” and “finally,” are notmeant to limit the scope of the present invention as the embodimentsdisclosed herein are merely exemplary.

The present invention relates to the generation of offers based uponcontracts that are agreed upon between the retailer and another party.Often the other party is a distributor or the manufacturer of theproduct. In this application the term “manufacturer” may include theactual producer of a good, or could include resellers or intermediatebranding entities. The term “retailer” refers to a business entity thatis offering the good or service to consumers directly, or less commonlyto yet another downstream business entity. Examples of manufacturerscould include, for example, a soda bottling plant, a consumer packagegood (CPG) producer, or a wholesale retailer. Examples of retailers, incontrast include main-street stores, such as Target, Safeway, Walmartand the like.

Historically, when a retailer and another party would contract for agiven promotion, the retailer would implement the contracted offeracross all applicable retail spaces. The present systems and methodsprovide an alternative structure, whereby offers that are likely to bemore successful, yet are similar to the contracted offer, and deployedin some subgroup of the retail locations, allowing for better testing ofoffer formats and types, as well as maximization of a retailer'sbusiness goals.

The following description of some embodiments will be provided inrelation to numerous subsections. The use of subsections, with headings,is intended to provide greater clarity and structure to the presentinvention. In no way are the subsections intended to limit or constrainthe disclosure contained therein. Thus, disclosures in any one sectionare intended to apply to all other sections, as is applicable.

I. Forward Looking Promotion Optimization

Within the forward-looking promotion optimization, revealed preferencesare obtained when the individual consumers respond to specificallydesigned actual test promotions. The revealed preferences may be trackedin individual computer-implemented accounts (which may, for example, beimplemented via a record in a centralized database and renderedaccessible to the merchant or the consumer via a computer network suchas the internet) associated with individual consumers, or may becollected at a physical retailer based upon transaction records. Forexample, when a consumer responds, using his smart phone, web browser,or in a physical store through completion of a transaction, to a testpromotion that offers 20% off a particular consumer packaged goods (CPG)item, that response is tracked in his individual computer-implementedaccount, or in a transaction record. Such computer-implemented accountsmay be implemented via, for example, a loyalty card program, apps on asmart phone, computerized records, social media news feed, etc.

In one or more embodiments, a plurality of test promotions may bedesigned and tested on a plurality of groups of consumers (the groups ofconsumers are referred to herein as “subpopulations”). The responses bythe consumers are recorded and analyzed, with the analysis resultemployed to generate additional test promotions or to formulate thegeneral population promotion. In the event of physical testing in aretailer space, it may be possible to segment the consumer base using avariety of collected demographic and activity data. This may include aloyalty care program, pharmacy ID, and information collected from publicdata sources. Such information may be correlated to credit card (ordebit card, electronic wallet, etc.) information, and stored as aprofile for the individual consumer and aggregated into consumerhouseholds. Information about the consumer, or the household, such aspurchasing behaviors, income levels, ethnicity, age(s), number ofpersons in the household, gender, political affiliations, geography,medical conditions, etc., may be used to categorize the consumers intolike subpopulations using neural network techniques and known clusteringalgorithms.

As will be discussed later herein, if the consumer actually redeems theoffer, one type of response is recorded and noted in thecomputer-implemented account of that consumer. Even if an action by theconsumer does not involve actually redeeming or actually takingadvantage of the promotional offer right away, an action by thatconsumer may, however, constitute a response that indicates a level ofinterest or lack of interest and may still be useful in revealing theconsumer preference (or lack thereof). For example, if a consumer savesan electronic coupon (offered as part of a test promotion) in hiselectronic coupon folder or forwards that coupon to a friend via anemail or a social website, that action may indicate a certain level ofinterest and may be useful in determining the effectiveness of a giventest promotion. In the physical retailer space, if a consumer stops tolook at a product, or even pick up the product but chooses not topurchase it at the register, such activity, to the extent it is reliablymeasured, may indicate interest in the promotion despite the lack of atransaction being completed. Different types of responses/actions by theconsumers may be accorded different weights, in one or more embodiments.

As noted, the groups of consumers involved in promotion testingrepresent segments of the public that have been purposefully segmentedin accordance with segmenting criteria specifically designed for thepurpose of testing the test promotions. As the term is employed herein,a subpopulation is deemed purposefully segmented when its members areselected based on criteria other than merely to make up a given numberof members in the subpopulation. Demographics, buying behavior,behavioral economics, geography (e.g., purchasing at a certain brick andmortar retailer) are example criteria that may be employed topurposefully segment a population into subpopulations for promotiontesting. In an example, a segmented population may number in the tens orhundreds or even thousands of individuals. In contrast, the generalpublic may involve tens of thousands, hundreds of thousands, or millionsof potential customers.

By purposefully segmenting the public into small subpopulations forpromotion testing, embodiments of the invention can exert control overvariables such as demographics (e.g., age, income, sex, marriage status,address, etc.), buying behavior (e.g., regular purchaser of Brand Xcookies, consumer of premium food, frequent traveler, etc.), weather,shopping habits, life style, and/or any other criteria suitable for usein creating the subpopulations. More importantly, the subpopulations arekept small such that multiple test promotions may be executed ondifferent subpopulations, either simultaneously or at different times,without undue cost or delay in order to obtain data pertaining to thetest promotion response behavior. The low cost/low delay aspect ofcreating and executing test promotions on purposefully segmentedsubpopulations permits, for example, what-if testing, testing instatistically significant numbers of tests, and/or iterative testing toisolate winning features in test promotions.

Generally speaking, each individual test promotion may be designed totest one or more test promotion variables. These test promotionsvariables may relate to, for example, the size, shape, color, manner ofdisplay, manner of discount, manner of publicizing, manner ofdissemination pertaining to the goods/services being promoted.

As a very simple example, one test promotion may involve 12-oz packagesof fancy-cut potato chips with medium salt and a discount of 30% off theregular price. This test promotion may be tested on a purposefullysegmented subpopulation of 35-40 years old professionals in the$30,000-$50,000 annual income range. Another test promotion may involvethe same 30% discount 12-oz packages of fancy-cut potato chips withmedium salt on a different purposefully segmented subpopulation of 35-40years old professionals in the higher $100,000-$150,000 annual incomerange. By controlling all variables except for income range, theresponses of these two test promotions, if repeated in statisticallysignificant numbers, would likely yield fairly accurate informationregarding the relationship between income for 35-40 years oldprofessionals and their actual preference for 12-oz packages of fancycut potato chips with medium salt.

In designing different test promotions, one or more of the testpromotions variables may vary or one or more of the segmenting criteriaemployed to create the purposefully segmented subpopulations may vary.The test promotion responses from individuals in the subpopulations arethen collected and analyzed to ascertain which test promotion or testpromotion variable(s) yields/yield the most desirable response (based onsome predefined success criteria, for example).

Further, the test promotions can also reveal insights regarding whichsubpopulation performs the best, or well, with respect to test promotionresponses. In this manner, test promotion response analysis providesinsights not only regarding the relative performance of the testpromotion and/or test promotion variable but also insights regardingpopulation segmentation and/or segmentation criteria. In an embodiment,it is contemplated that the segments may be arbitrarily or randomlysegmented into groups and test promotions may be executed against thesearbitrarily segmented groups in order to obtain insights regardingpersonal characteristics that respond well to a particular type ofpromotion.

In an embodiment, the identified test promotion variable(s) that yieldthe most desirable responses may then be employed to formulate a generalpublic promotion (GPP), which may then be offered to the larger public.A general public promotion is different from a test promotion in that ageneral public promotion is a promotion designed to be offered tomembers of the public to increase or maximize sales or profit whereas atest promotion is designed to be targeted to a small group ofindividuals fitting a specific segmentation criteria for the purpose ofpromotion testing. Examples of general public promotions include (butnot limited to) advertisement printed in newspapers, release in publicforums and websites, flyers for general distribution, announcement onradios or television, promotion broadly transmitted or made available tomembers of the public, and/or promotions that are rolled out to a widerset of physical retailer locations. The general public promotion maytake the form of a paper or electronic circular that offers the samepromotion to the larger public, for example.

Alternatively or additionally, promotion testing may be iterated overand over with different subpopulations (segmented using the same ordifferent segmenting criteria) and different test promotions (devisedusing the same or different combinations of test promotion variables) inorder to validate one or more the test promotion response analysisresult(s) prior to the formation of the generalized public promotion. Inthis manner, “false positives” may be reduced.

Since a test promotion may involve many test promotion variables,iterative test promotion testing, as mentioned, may help pin-point avariable (e.g., promotion feature) that yields the most desirable testpromotion response to a particular subpopulation or to the generalpublic.

Suppose, for example, that a manufacturer wishes to find out the mosteffective test promotion for packaged potato chips. One test promotionmay reveal that consumers tend to buy a greater quantity of potato chipswhen packaged in brown paper bags versus green paper bags. That“winning” test promotion variable value (e.g., brown paper bagpackaging) may be retested in another set of test promotions usingdifferent combinations of test promotion variables (such as for examplewith different prices, different display options, etc.) on the same ordifferent purposefully segmented subpopulations. The follow-up testpromotions may be iterated multiple times in different test promotionvariable combinations and/or with different test subpopulations tovalidate that there is, for example, a significant consumer preferencefor brown paper bag packaging over other types of packaging for potatochips.

Further, individual “winning” test promotion variable values fromdifferent test promotions may be combined to enhance the efficacy of thegeneral public promotion to be created. For example, if a 2-for-1discount is found to be another winning variable value (e.g., consumerstend to buy a greater quantity of potato chips when offered a 2-for-1discount), that winning test promotion variable value (e.g., theaforementioned 2-for-1 discount) of the winning test promotion variable(e.g., discount depth) may be combined with the brown paper packagingwinning variable value to yield a promotion that involves discounting2-for-1 potato chips in brown paper bag packaging.

The promotion involving discounting 2-for-1 potato chips in brown paperbag packaging may be tested further to validate the hypothesis that sucha combination elicits a more desirable response than the response fromtest promotions using only brown paper bag packaging or from testpromotions using only 2-for-1 discounts. As many of the “winning” testpromotion variable values may be identified and combined in a singlepromotion as desired. At some point, a combination of “winning” testpromotion variables (involving one, two, three, or more “winning” testpromotion variables) may be employed to create the general publicpromotion, in one or more embodiments.

In one or more embodiments, test promotions may be executed iterativelyand/or in a continual fashion on different purposefully segmentedsubpopulations using different combinations of test promotion variablesto continue to obtain insights into consumer actual revealedpreferences, even as those preferences change over time. Note that theconsumer responses that are obtained from the test promotions are actualrevealed preferences instead of stated preferences. In other words, thedata obtained from the test promotions administered in accordance withembodiments of the invention pertains to what individual consumersactually do when presented with the actual promotions. The data istracked and available for analysis and/or verification in individualcomputer-implemented accounts of individual consumers involved in thetest promotions. This revealed preference approach is opposed to astated preference approach, which stated preference data is obtainedwhen the consumer states what he would hypothetically do in response to,for example, a hypothetically posed conjoint test question.

As such, the actual preference test promotion response data obtained inaccordance with embodiments of the present invention is a more reliableindicator of what a general population member may be expected to behavewhen presented with the same or a similar promotion in a general publicpromotion. Accordingly, there is a closer relationship between the testpromotion response behavior (obtained in response to the testpromotions) and the general public response behavior when a generalpublic promotion is generated based on such test promotion responsedata.

Also, the lower face validity of a stated preference test, even if theinsights have statistical relevance, poses a practical challenge; CPGmanufacturers who conduct such tests have to then communicate theinsights to a retailer in order to drive real-world behavior, andconvincing retailers of the validity of these tests after the fact canlead to lower credibility and lower adoption, or “signal loss” as thetop concepts from these tests get re-interpreted by a third party, theretailer, who wasn't involved in the original test design.

It should be pointed out that embodiments of the inventive testpromotion optimization methods and apparatuses disclosed herein operateon a forward-looking basis in that the plurality of test promotions aregenerated and tested on segmented subpopulations in advance of theformulation of a general public promotion. In other words, the analysisresults from executing the plurality of test promotions on differentpurposefully segmented subpopulations are employed to generate futuregeneral public promotions. In this manner, data regarding the “expected”efficacy of the proposed general public promotion is obtained evenbefore the proposed general public promotion is released to the public.This is one key driver in obtaining highly effective general publicpromotions at low cost.

Furthermore, the subpopulations can be generated with highly granularsegmenting criteria, allowing for control of data noise that may arisedue to a number of factors, some of which may be out of the control ofthe manufacturer or the merchant. This is in contrast to the aggregateddata approach of the prior art.

For example, if two different test promotions are executed on twosubpopulations shopping at the same merchant on the same date,variations in the response behavior due to time of day or trafficcondition are essentially eliminated or substantially minimized in theresults (since the time or day or traffic condition would affect the twosubpopulations being tested in substantially the same way).

The test promotions themselves may be formulated to isolate specifictest promotion variables (such as the aforementioned potato chip brownpaper packaging or the 16-oz size packaging). This is also in contrastto the aggregated data approach of the prior art.

Accordingly, individual winning promotion variables may be isolated andcombined to result in a more effective promotion campaign in one or moreembodiments. Further, the test promotion response data may be analyzedto answer questions related to specific subpopulation attribute(s) orspecific test promotion variable(s). With embodiments of the invention,it is now possible to answer, from the test subpopulation response data,questions such as “How deep of a discount is required to increase by 10%the volume of potato chip purchased by buyers who are 18-25 year-oldmale shopping on a Monday?” or to generate test promotions specificallydesigned to answer such a question. Such data granularity and analysisresult would have been impossible to achieve using the backward-looking,aggregate historical data approach of the prior art.

In one or more embodiments, there is provided a promotional idea modulefor generating ideas for promotional concepts to test. The promotionalidea generation module relies on a series of pre-constructed sentencestructures that outline typical promotional constructs. For example, BuyX, get Y for $Z price would be one sentence structure, whereas Get Y for$Z when you buy X would be a second. It's important to differentiatethat the consumer call to action in those two examples is materiallydifferent, and one cannot assume the promotional response will be thesame when using one sentence structure vs. another. The solution isflexible and dynamic, so once X, Y, and Z are identified, multiple validsentence structures can be tested. Additionally, other variables in thesentence could be changed, such as replacing “buy” with “hurry up andbuy” or “act now” or “rush to your local store to find”. The solutiondelivers a platform where multiple products, offers, and different waysof articulating such offers can be easily generated by a lay user. Theamount of combinations to test can be infinite. Further, the generationmay be automated, saving time and effort in generating promotionalconcepts. In following sections one mechanism, the design matrix, forthe automation of promotional generation will be provided in greaterdetail.

In one or more embodiments, once a set of concepts is developed, thetechnology advantageously a) will constrain offers to only test “viablepromotions”, e.g., those that don't violate local laws, conflict withbranding guidelines, lead to unprofitable concepts that wouldn't bepractically relevant, can be executed on a retailers' system, etc.,and/or b) link to the design of experiments for micro-testing todetermine which combinations of variables to test at any given time.

In one or more embodiments, there is provided an offer selection modulefor enabling a non-technical audience to select viable offers for thepurpose of planning traditional promotions (such as general populationpromotion, for example) outside the test environment. By using filtersand advanced consumer-quality graphics, the offer selection module willbe constrained to only show top performing concepts from the tests, withproduction-ready artwork wherever possible. By doing so, the offerselection module renders irrelevant the traditional, Excel-based orheavily numbers-oriented performance reports from traditional analytictools. The user can have “freedom within a framework” by selecting anyof the pre-scanned promotions for inclusion in an offer to the generalpublic, but value is delivered to the retailer or manufacturer becausethe offers are constrained to only include the best performing concepts.Deviation from the top concepts can be accomplished, but only once thespecific changes are run through the testing process and emerge in theoffer selection windows.

In an embodiment, it is expressly contemplated that the generalpopulation and/or subpopulations may be chosen from social media site(e.g., Facebook™, Twitter™, Google+™, etc.) participants. Social mediaoffers a large population of active participants and often providevarious communication tools (e.g., email, chat, conversation streams,running posts, etc.) which makes it efficient to offer promotions and toreceive responses to the promotions. Various tools and data sourcesexist to uncover characteristics of social media site members, whichcharacteristics (e.g., age, sex, preferences, attitude about aparticular topic, etc.) may be employed as highly granular segmentationcriteria, thereby simplifying segmentation planning.

Although grocery stores and other brick-and-mortar businesses arediscussed in various examples herein, it is expressly contemplated thatembodiments of the invention apply also to online shopping and onlineadvertising/promotion and online members/customers.

These and other features and advantages of embodiments of the inventionmay be better understood with reference to the figures and discussionsthat follow.

FIG. 2A shows, in accordance with an embodiment of the invention, aconceptual drawing of the forward-looking promotion optimization method.As shown in

FIG. 2A, a plurality of test promotions 102 a, 102 b, 102 c, 102 d, and102 e are administered to purposefully segmented subpopulations 104 a,104 b, 104 c, 104 d, and 104 e respectively. As mentioned, each of thetest promotions (102 a-102 e) may be designed to test one or more testpromotion variables.

In the example of FIG. 2A, test promotions 102 a-102 d are shown testingthree test promotion variables X, Y, and Z, which may represent forexample the size of the packaging (e.g., 12 oz versus 16 oz), the mannerof display (e.g., at the end of the aisle versus on the shelf), and thediscount (e.g., 10% off versus 2-for-1). These promotion variables areof course only illustrative and almost any variable involved inproducing, packaging, displaying, promoting, discounting, etc. of thepackaged product may be deemed a test promotion variable if there is aninterest in determining how the consumer would respond to variations ofone or more of the test promotion variables. Further, although only afew test promotion variables are shown in the example of FIG. 2A, a testpromotion may involve as many or as few of the test promotion variablesas desired. For example, test promotion 102 e is shown testing four testpromotion variables (X, Y, Z, and T).

One or more of the test promotion variables may vary from test promotionto test promotion. In the example of FIG. 2A, test promotion 102 ainvolves test variable X1 (representing a given value or attribute fortest variable X) while test promotion 102 b involves test variable X2(representing a different value or attribute for test variable X). Atest promotion may vary, relative to another test promotion, one testpromotion variable (as can be seen in the comparison between testpromotions 102 a and 102 b) or many of the test promotion variables (ascan be seen in the comparison between test promotions 102 a and 102 d).Also, there are no requirements that all test promotions must have thesame number of test promotion variables (as can be seen in thecomparison between test promotions 102 a and 102 e) although for thepurpose of validating the effect of a single variable, it may be usefulto keep the number and values of other variables (e.g., the controlvariables) relatively constant from test to test (as can be seen in thecomparison between test promotions 102 a and 102 b).

Generally speaking, the test promotions may be generated using automatedtest promotion generation software 110, which varies for example thetest promotion variables and/or the values of the test promotionvariables and/or the number of the test promotion variables to come upwith different test promotions.

In the example of FIG. 2A, purposefully segmented subpopulations 104a-104 d are shown segmented using four segmentation criteria A, B, C, D,which may represent for example the age of the consumer, the householdincome, the zip code, group of consumers shopping at a particularphysical retailer, and whether the person is known from past purchasingbehavior to be a luxury item buyer or a value item buyer. Thesesegmentation criteria are of course only illustrative and almost anydemographics, behavioral, attitudinal, whether self-described,objective, interpolated from data sources (including past purchase orcurrent purchase data), etc. may be used as segmentation criteria ifthere is an interest in determining how a particular subpopulation wouldlikely respond to a test promotion. Further, although only a fewsegmentation criteria are shown in connection with subpopulations 104a-104 d in the example of FIG. 2A, segmentation may involve as many oras few of the segmentation criteria as desired. For example,purposefully segmented subpopulation 104 e is shown segmented using fivesegmentation criteria (A, B, C, D, and E).

In the present disclosure, a distinction is made between a purposefullysegmented subpopulation and a randomly segmented subpopulation. Theformer denotes a conscious effort to group individuals based on one ormore segmentation criteria or attributes. The latter denotes a randomgrouping for the purpose of forming a group irrespective of theattributes of the individuals. Randomly segmented subpopulations areuseful in some cases; however they are distinguishable from purposefullysegmented subpopulations when the differences are called out.

One or more of the segmentation criteria may vary from purposefullysegmented subpopulation to purposefully segmented subpopulation. In theexample of FIG. 2A, purposefully segmented subpopulation 104 a involvessegmentation criterion value A1 (representing a given attribute or rangeof attributes for segmentation criterion A) while purposefully segmentedsubpopulation 104 c involves segmentation criterion value A2(representing a different attribute or set of attributes for the samesegmentation criterion A).

As can be seen, different purposefully segmented subpopulation may havedifferent numbers of individuals. As an example, purposefully segmentedsubpopulation 104 a has four individuals (P1-P4) whereas purposefullysegmented subpopulation 104 e has six individuals (P17-P22). Apurposefully segmented subpopulation may differ from anotherpurposefully segmented subpopulation in the value of a singlesegmentation criterion (as can be seen in the comparison betweenpurposefully segmented subpopulation 104 a and purposefully segmentedsubpopulation 104 c wherein the attribute A changes from A1 to A2) or inthe values of many segmentation criteria simultaneously (as can be seenin the comparison between purposefully segmented subpopulation 104 a andpurposefully segmented subpopulation 104 d wherein the values forattributes A, B, C, and D are all different). Two purposefully segmentedsubpopulations may also be segmented identically (e.g., using the samesegmentation criteria and the same values for those criteria) as can beseen in the comparison between purposefully segmented subpopulation 104a and purposefully segmented subpopulation 104 b.

Also, there are no requirements that all purposefully segmentedsubpopulations must be segmented using the same number of segmentationcriteria (as can be seen in the comparison between purposefullysegmented subpopulation 104 a and 104 e wherein purposefully segmentedsubpopulation 104 e is segmented using five criteria and purposefullysegmented subpopulation 104 a is segmented using only four criteria)although for the purpose of validating the effect of a single criterion,it may be useful to keep the number and values of other segmentationcriteria (e.g., the control criteria) relatively constant frompurposefully segmented subpopulation to purposefully segmentedsubpopulation.

Generally speaking, the purposefully segmented subpopulations may begenerated using automated segmentation software 112, which varies forexample the segmentation criteria and/or the values of the segmentationcriteria and/or the number of the segmentation criteria to come up withdifferent purposefully segmented subpopulations.

In one or more embodiments, the test promotions are administered toindividual users in the purposefully segmented subpopulations in such away that the responses of the individual users in that purposefullysegmented subpopulation can be recorded for later analysis. As anexample, an electronic coupon may be presented in an individual user'scomputer-implemented account (e.g., shopping account or loyaltyaccount), or emailed or otherwise transmitted to the smart phone of theindividual. In an example, the user may be provided with an electroniccoupon on his smart phone that is redeemable at the merchant. In FIG.2A, this administering is represented by the lines that extend from testpromotion 102 a to each of individuals P1-P4 in purposefully segmentedsubpopulation 104 a. If the user (such as user P1) makes apromotion-significant response, the response is noted in database 130.

A promotion-significant response is defined as a response that isindicative of some level of interest or disinterest in the goods/servicebeing promoted. In the aforementioned example, if the user P1 redeemsthe electronic coupon at the store, the redemption is stronglyindicative of user P1's interest in the offered goods. However,responses falling short of actual redemption or actual purchase maystill be significant for promotion analysis purposes. For example, ifthe user saves the electronic coupon in his electronic coupon folder onhis smart phone, such action may be deemed to indicate a certain levelof interest in the promoted goods. As another example, if the userforwards the electronic coupon to his friend or to a social networksite, such forwarding may also be deemed to indicate another level ofinterest in the promoted goods. As another example, if the user quicklymoves the coupon to trash, this action may also indicate a level ofstrong disinterest in the promoted goods. In one or more embodiments,weights may be accorded to various user responses to reflect the levelof interest/disinterest associated with the user's responses to a testpromotion. For example, actual redemption may be given a weight of 1,whereas saving to an electronic folder would be given a weight of only0.6 and whereas an immediate deletion of the electronic coupon would begiven a weight of −0.5.

Analysis engine 132 represents a software engine for analyzing theconsumer responses to the test promotions. Response analysis may employany analysis technique (including statistical analysis) that may revealthe type and degree of correlation between test promotion variables,subpopulation attributes, and promotion responses. Analysis engine 132may, for example, ascertain that a certain test promotion variable value(such as 2-for-1 discount) may be more effective than another testpromotion variable (such as 25% off) for 32-oz soft drinks if presentedas an electronic coupon right before Monday Night Football. Suchcorrelation may be employed to formulate a general population promotion(150) by a general promotion generator software (160). As can beappreciated from this discussion sequence, the optimization is aforward-looking optimization in that the results from test promotionsadministered in advance to purposefully segmented subpopulations areemployed to generate a general promotion to be released to the public ata later date.

In one or more embodiments, the correlations ascertained by analysisengine 132 may be employed to generate additional test promotions(arrows 172, 174, and 176) to administer to the same or a different setof purposefully segmented subpopulations. The iterative testing may beemployed to verify the consistency and/or strength of a correlation (byadministering the same test promotion to a different purposefullysegmented subpopulation or by combining the “winning” test promotionvalue with other test promotion variables and administering there-formulated test promotion to the same or a different set ofpurposefully segmented subpopulations).

In one or more embodiments, a “winning” test promotion value (e.g., 20%off listed price) from one test promotion may be combined with another“winning” test promotion value (e.g., packaged in plain brown paperbags) from another test promotion to generate yet another testpromotion. The test promotion that is formed from multiple “winning”test promotion values may be administered to different purposefullysegmented subpopulations to ascertain if such combination would eliciteven more desirable responses from the test subjects.

Since the purposefully segmented subpopulations are small and may besegmented with highly granular segmentation criteria, a large number oftest promotions may be generated (also with highly granular testpromotion variables) and a large number of combinations of testpromotions/purposefully segmented subpopulations can be executed quicklyand at a relatively low cost. The same number of promotions offered asgeneral public promotions would have been prohibitively expensive toimplement, and the large number of failed public promotions would havebeen costly for the manufacturers/retailers. In contrast, if a testpromotion fails, the fact that the test promotion was offered to only asmall number of consumers in one or more segmented subpopulations, or alimited number of physical locations for a limited time, would limit thecost of failure. Thus, even if a large number of these test promotions“fail” to elicit the desired responses, the cost of conducting thesesmall test promotions would still be quite small.

In an embodiment, it is envisioned that dozens, hundreds, or eventhousands of these test promotions may be administered concurrently orstaggered in time to the dozens, hundreds or thousands of segmentedsubpopulations. Further, the large number of test promotions executed(or iteratively executed) improves the statistical validity of thecorrelations ascertained by analysis engine. This is because the numberof variations in test promotion variable values, subpopulationattributes, etc. can be large, thus yielding rich and granulated resultdata. The data-rich results enable the analysis engine to generatehighly granular correlations between test promotion variables,subpopulation attributes, and type/degree of responses, as well as trackchanges over time. In turn, these more accurate/granular correlationshelp improve the probability that a general public promotion createdfrom these correlations would likely elicit the desired response fromthe general public. It would also, over, time, create promotionalprofiles for specific categories, brands, retailers, and individualshoppers where, e.g., shopper 1 prefers contests and shopper 2 prefersinstant financial savings.

FIG. 2B shows, in accordance with an embodiment of the invention, thesteps for generating a general public promotion. In one or moreembodiments, each, some, or all the steps of FIG. 2B may be automatedvia software to automate the forward-looking promotion optimizationprocess. In step 202, the plurality of test promotions are generated.These test promotions have been discussed in connection with testpromotions 102 a-102 e of FIG. 2A and represent the plurality of actualpromotions administered to small purposefully segmented subpopulationsto allow the analysis engine to uncover highly accurate/granularcorrelations between test promotion variables, subpopulation attributes,and type/degree of responses in an embodiment, these test promotions maybe generated using automated test promotion generation software thatvaries one or more of the test promotion variables, either randomly,according to heuristics, and/or responsive to hypotheses regardingcorrelations from analysis engine 132 for example.

In step 204, the segmented subpopulations are generated. In anembodiment, the segmented subpopulations represent randomly segmentedsubpopulations. In another embodiment, the segmented subpopulationsrepresent purposefully segmented subpopulations. In another embodiment,the segmented subpopulations may represent a combination of randomlysegmented subpopulations and purposefully segmented subpopulations. Inan embodiment, these segmented subpopulations may be generated usingautomated subpopulation segmentation software that varies one or more ofthe segmentation criteria, either randomly, according to heuristics,and/or responsive to hypotheses regarding correlations from analysisengine 132, for example.

In step 206, the plurality of test promotions generated in step 202 areadministered to the plurality of segmented subpopulations generated instep 204. In an embodiment, the test promotions are administered toindividuals within the segmented subpopulation and the individualresponses are obtained and recorded in a database (step 208).

In an embodiment, automated test promotion software automaticallyadministers the test promotions to the segmented subpopulations usingelectronic contact data that may be obtained in advance from, forexample, social media sites, a loyalty card program, previous contactwith individual consumers, or potential consumer data purchased from athird party, etc. In some alternate embodiments, as will be discussed ingreater detail below, the test promotions may be administered viaelectronic pricing tags displayed within a physical retail location.Such physical test promotions may be constricted by deployment time dueto logistic considerations. The responses may be obtained at the pointof sale terminal, or via a website or program, via social media, or viaan app implemented on smart phones used by the individuals, for example.

In step 210, the responses are analyzed to uncover correlations betweentest promotion variables, subpopulation attributes, and type/degree ofresponses.

In step 212, the general public promotion is formulated from thecorrelation data, which is uncovered by the analysis engine from dataobtained via subpopulation test promotions. In an embodiment, thegeneral public promotion may be generated automatically using publicpromotion generation software which utilizes at least the test promotionvariables and/or subpopulation segmentation criteria and/or test subjectresponses and/or the analysis provided by analysis engine 132.

In step 214, the general public promotion is released to the generalpublic to promote the goods/services.

In one or more embodiments, promotion testing using the test promotionson the segmented subpopulations occurs in parallel to the release of ageneral public promotion and may continue in a continual fashion tovalidate correlation hypotheses and/or to derive new general publicpromotions based on the same or different analysis results. If iterativepromotion testing involving correlation hypotheses uncovered by analysisengine 132 is desired, the same test promotions or new test promotionsmay be generated and executed against the same segmented subpopulationsor different segmented subpopulations as needed (paths 216/222/226 or216/224/226 or 216/222/224/226). As mentioned, iterative promotiontesting may validate the correlation hypotheses, serve to eliminate“false positives” and/or uncover combinations of test promotionvariables that may elicit even more favorable or different responsesfrom the test subjects.

Promotion testing may be performed on an on-going basis using the sameor different sets of test promotions on the same or different sets ofsegmented subpopulations as mentioned (paths 218/222/226 or 218/224/226or 218/222/224/226 or 220/222/226 or 220/224/226 or 220/222/224/226).

FIG. 3A shows in greater detail, in accordance with an embodiment of theinvention, the administering step 206 of FIG. 2 from the user'sperspective. In step 302, the test promotion is received from the testpromotion generation server (which executes the software employed togenerate the test promotion). As examples, the test promotion may bereceived at a user's smart phone or tablet (such as in the case of anelectronic coupon or a discount code, along with the associatedpromotional information pertaining to the product, place of sale, timeof sale, etc.), in a computer-implemented account (such as a loyaltyprogram account) associated with the user that is a member of thesegmented subpopulation to be tested, via one or more social mediasites, or displayed on electronic pricing tags within a retailer'sphysical store. In step 304, the test promotion is presented to theuser. In step 306, the user's response to the test promotion is obtainedand transmitted to a database for analysis.

FIG. 3B shows in greater detail, in accordance with an embodiment of theinvention, the administering step 206 of FIG. 2 from the forward-lookingpromotion optimization system perspective. In step 312, the testpromotions are generated using the test promotion generation server(which executes the software employed to generate the test promotion).In step 314, the test promotions are provided to the users (e.g.,transmitted or emailed to the user's smart phone or tablet or computer,shared with the user using the user's loyalty account, displayed in thephysical retailer). In step 316, the system receives the user'sresponses and stores the user's responses in the database for lateranalysis.

FIG. 4 shows various example segmentation criteria that may be employedto generate the purposefully segmented subpopulations. As show in FIG.4, demographics criteria (e.g., sex, location, household size, householdincome, etc.), buying behavior (category purchase index, most frequentshopping hours, value versus premium shopper, etc.), past/currentpurchase history, channel (e.g., stores frequently shopped at,competitive catchment of stores within driving distance), behavioraleconomics factors, etc. can all be used to generate with a high degreeof granularity the segmented subpopulations. The examples of FIG. 4 aremeant to be illustrative and not meant to be exhaustive or limiting. Asmentioned, one or more embodiments of the invention generate thesegmented subpopulations automatically using automated populationsegmentation software that generates the segmented subpopulations basedon values of segmentation criteria.

FIG. 5 shows various example methods for communicating the testpromotions to individuals of the segmented subpopulations being tested.As shown in FIG. 5, the test promotions may be mailed to theindividuals, emailed in the form of text or electronic flyer or couponor discount code, displayed on a webpage when the individual accesseshis shopping or loyalty account via a computer or smart phone or tablet,and lastly display on an electronic pricing tag within a retailer'sstore. Redemption may take place using, for example, a printed coupon(which may be mailed or may be printed from an electronic version of thecoupon) at the point of sale terminal, an electronic version of thecoupon (e.g., a screen image or QR code), the verbal providing or manualentry of a discount code into a terminal at the store or at the point ofsale, or purchase of an item in a physical location that has thepromotion displayed. The examples of FIG. 5 are meant to be illustrativeand not meant to be exhaustive or limiting. One or more embodiments ofthe invention automatically communicate the test promotions toindividuals in the segmented subpopulations using software thatcommunicates/email/mail/administer the test promotions automatically. Inthis manner, subpopulation test promotions may be administeredautomatically, which gives manufacturers and retailers the ability togenerate and administer a large number of test promotions with lowcost/delay.

FIG. 6 shows, in accordance with an embodiment, various examplepromotion-significant responses. As mentioned, redemption of the testoffer is one strong indication of interest in the promotion. However,other consumer actions responsive to the receipt of a promotion may alsoreveal the level of interest/disinterest and may be employed by theanalysis engine to ascertain which test promotion variable is likely orunlikely to elicit the desired response. Examples shown in FIG. 6include redemption (strong interest), deletion of the promotion offer(low interest), save to electronic coupon folder (mild to stronginterest), clicked to read further (mild interest), forwarding to selfor others or social media sites (mild to strong interest), stopping tolook at an item within the store (mild interest), and picking up theitem in a physical store but ultimately not purchasing the item (stronginterest). As mentioned, weights may be accorded to various consumerresponses to allow the analysis engine to assign scores and provideuser-interest data for use in formulating follow-up test promotionsand/or in formulating the general public promotion. For example, lowinterest may be afforded a score of −0.75 to −0.25, mild interest couldbe afforded a score weight of 0.1-0.5, strong interest may be afforded ascore of 0.5-0.8, and purchase of the product may be afforded a scoreof 1. The examples of FIG. 6 are meant to be illustrative and not meantto be exhaustive or limiting.

FIG. 7 shows, in accordance with an embodiment of the invention, variousexample test promotion variables affecting various aspects of a typicaltest promotion. As shown in FIG. 7, example test promotion variablesinclude price, discount action (e.g., save 10%, save $1, 2-for-1 offer,etc.), artwork (e.g., the images used in the test promotion to drawinterest), brand (e.g., brand X potato chips versus brand Y potatochips), pricing tier (e.g., premium, value, economy), size (e.g., 32 oz,16 oz, 8 oz), packaging (e.g., single, 6-pack, 12-pack, paper, can,etc.), channel (e.g., email versus paper coupon versus notification inloyalty account). The examples of FIG. 7 are meant to be illustrativeand not meant to be exhaustive or limiting. As mentioned, one or moreembodiments of the invention involve generating the test promotionsautomatically using automated test promotion generation software byvarying one or more of the test promotion variables, either randomly orbased on feedback from the analysis of other test promotions or from theanalysis of the general public promotion.

FIG. 8 shows, in accordance with an embodiment of the invention, ageneral hardware/network view of the forward-looking promotionoptimization system 800. In general, the various functions discussed maybe implemented as software modules, which may be implemented in one ormore servers (including actual and/or virtual servers). In FIG. 8, thereis shown a test promotion generation module 802 for generating the testpromotions in accordance with test promotion variables. There is alsoshown a population segmentation module 804 for generating the segmentedsubpopulations in accordance with segmentation criteria. There is alsoshown a test promotion administration module 806 for administering theplurality of test promotions to the plurality of segmentedsubpopulations. There is also shown an analysis module 808 for analyzingthe responses to the test promotions as discussed earlier. There is alsoshown a general population promotion generation module 810 forgenerating the general population promotion using the analysis result ofthe data from the test promotions. There is also shown a module 812,representing the software/hardware module for receiving the responses.Module 812 may represent, for example, the point of sale terminal in astore, a shopping basket on an online shopping website, an app on asmart phone, a webpage displayed on a computer, a social media newsfeed, etc. where user responses can be received.

One or more of modules 802-812 may be implemented on one or moreservers, as mentioned. A database 814 is shown, representing the datastore for user data and/or test promotion and/or general publicpromotion data and/or response data. Database 814 may be implemented bya single database or by multiple databases. The servers and database(s)may be coupled together using a local area network, an intranet, theinterne, or any combination thereof (shown by reference number 830).

User interaction for test promotion administration and/or acquiring userresponses may take place via one or more of user interaction devices.Examples of such user interaction devices are wired laptop 840, wiredcomputer 844, wireless laptop 846, wireless smart phone or tablet 848.Test promotions may also be administered via printing/mailing module850, which communicates the test promotions to the users via mailings852 or printed circular 854. The example components of FIG. 8 are onlyillustrative and are not meant to be limiting of the scope of theinvention. The general public promotion, once generated, may also becommunicated to the public using some or all of the user interactiondevices/methods discussed herein.

As can be appreciated by those skilled in the art, providing aresult-effective set of recommendations for a generalized publicpromotion is one of the more important tasks in test promotionoptimization.

In one or more embodiments, there are provided adaptive experimentationand optimization processes for automated promotion testing. Testing issaid to be automated when the test promotions are generated in themanner that is likely produce the desired response consistent with thegoal of the generalized public promotion.

For example, if the goal is to maximize profit for the sale of a certainnewly created brand of potato chips, embodiments of the inventionoptimally and adaptively, without using required human intervention,plan the test promotions, iterate through the test promotions to testthe test promotion variables in the most optimal way, learn and validatesuch that the most result-effective set of test promotions can bederived, and provide such result-effective set of test promotions asrecommendations for generalized public promotion to achieve the goal ofmaximizing profit for the sale of the newly created brand of potatochips.

The term “without required human intervention” does not denote zerohuman intervention. The term however denotes that the adaptiveexperimentation and optimization processes for automated promotiontesting can be executed without human intervention if desired. However,embodiments of the invention do not exclude the optional participationof humans, especially experts, in various phases of the adaptiveexperimentation and optimization processes for automated promotiontesting if such participation is desired at various points to injecthuman intelligence or experience or timing or judgment in the adaptiveexperimentation and optimization processes for automated promotiontesting process. Further, the term does not exclude the optionalnonessential ancillary human activities that can otherwise also beautomated (such as issuing the “run” command to begin generating testpromotions or issuing the “send” command to send recommendationsobtained).

II. Offer Generation

As noted previously, offers may be presented by the retailer effectivelyas they have control over the retail space, and being furthest along inthe supply chain, typically have more margin available to makemeaningful discounts. Likewise, they can collect transaction logs fordownstream analysis of the offer effectiveness. However, while retailersare excellent at selling goods, they typically lack the backendinfrastructure and expertise to properly generate, administer andanalyze a comprehensive promotional campaign.

Additionally, manufacturers and distributors themselves have a stronginterest in having their specific products promoted (especially overtheir competitor's products). To this end, manufacturers often engageretailers in promotion agreements. These contracts are usually for agiven time period, designate one or more products for discount, and theterms of the offer. Historically, the retailers compile these contractedpromotions and roll them out, carte blanche, to all applicable retailoutlets. While this may result in increased sales (or other objective),it typically is suboptimal, and could be improved upon by applyingimproved promotion strategies.

In order to improve the prior methods of contracted offer delivery andredemption, systems and methods of an offer generation tool based uponcontracted offers, and associated backend systems, are provided. Tofacilitate the discussion, FIG. 9 provides an example block diagram ofan offer generation architecture 900. A plurality of retailers 910 a-xmay interact with a plurality of manufacturers (or distributors) 920 a-yusing an offer system 960 intermediary. This offer system 960 replacesthe activities typically performed by a third party consultant, butdelivers more refined and accurate results at a much lower cost thantraditional methods. The offer system 960 may access one or more datastores 980, which may be populated with offer structures from an offerbank from the offer system 960, transaction logs from the retailers 910a-x, contracts that are active, product details from the manufacturers920 a-y and additional third party data (such as trade group analyticaldata).

The various components of the offer generation architecture communicatewith one another via a network 950. This network 950 may include theinternet, cellular network(s), private or municipal local and wide areanetworks, or any combination thereof. In some particular embodiments,persons employed at the retailer and manufacturer may access the offersystem on a web based application via the internet and a web browser.The offer system 960 received inputs from the retailer regarding thecontracts for product promotions. Data from the contracts, includingcontract start and duration, product(s) implicated, and offer structureare gleaned from the contracts.

The system can then access an offer bank and select offers, usingmachine learning algorithms, that are designed to best address theobjective for the promotional campaign. The objective may be a defaultof profit or may be configured by the retailer directly. Objectivestypically include profitability, gross revenue, product margin, or salesvolumes.

The optimal offers are then scored against the contracted offer details.One example of possible scoring metrics are provided in greater detailin relation to A attached hereto. Essentially, the scoring algorithmlooks at various features of the proposed offers, and penalizes then asthey migrate too far from the contracted offer. In this manner a smallselect (typically 2-5) test offers are selected from the larger listingof optimized offers. These test offers, plus the contracted offer, areall deployed at various retailer locations, and transaction logs fromthe offer duration period are collected for further refinement of theoffer bank.

The offer system 960 is presented in greater detail at FIG. 10. Asnoted, the data store(s) 980 includes, at a minimum, transaction logs981 collected from one or more retailers, a database of the contractsthe retailer is engaged in 983, product data 985 from the manufacturers,and a repository of prior offers 987. The offer repository 987 is anoffer bank of existing offers and analytics regarding offer performance.Analytics may be stored along with the offer for a given populationbase, and if already deployed previously, the actual response results.This saves significant computational resources, which improves theserver's operation by delivering meaningful results with minimaladditional computational load.

The offer system itself 960 includes a contract parser 963 that uses thecontracts database to select contracts for offer testing. The parser mayleverage optical character recognition (OCR) techniques, naturallanguage processing (NLP) and trained machine learning models to extractrelevant information from the contracts directly. Alternatively, thecontract parser may include a human interface that allows an operator toextract the pertinent information from the contract directly. In yetother embodiments, the system may utilize a hybrid approach, wherebyconfidence of the automated data extraction is measured, and if below athreshold, a human is requested to intervene and process the relevantcontract. Information typically extracted from the contract includes thestart date, duration of the offer, product(s) implicated by the offer,and offer structure and amounts.

The offer optimizer 967 leverages the old transaction logs 981, productdata 985 and objectives, to generate optimal offers (as discussed in theabove section in considerable detail). These optimal offers are storedin the offer database 987 as an offer bank for later usage. In someembodiments, the offer optimizations leverage one or more machinelearning models to generate the optimal models.

The offer manager 965 works in conjunction with the retailer to definean objective of the offer, and ultimately assists in the administrationof the offer via the offer portal 961. As the objective is received theoffer optimizer 967 may select offers from the offer bank that are bestsuited to implement along with the contracted offer. Offers areallocated according to their probability of outperforming other offerson the customers objective using reinforced learning techniques. Thenumber of each offer to run in the market is chosen using reinforcedlearning techniques, such as Thompson sampling, and then offers are thenallocated to maximize orthogonality between a set of variables. Thesevariables may include store sales, store stock-out rates, number ofrelevant SKUs carried in each store, temporal effects and offerfeatures. Offer features can include discount depth, buy quantity andoffer structure. The machine learning model performs an optimizationproblem in which model predictions and uncertainty are computed for eachoffer for each store for each period and offers are allocated tomaximize customer objectives while reducing model uncertainty. Thestatistical/machine learning model shares information collected abouteach offer, so that offers are not considered independently. Offers maylikewise be scored against the contracted offer to select the set offinal test offers that are to be deployed. The scoring process looks atvarious components of the proposed offers against the contracted offer.As the proposed offers deviate from the contracted offer, they may bepenalized.

Using the offer scores, and anticipated forecast of completing theobjective, a select few (configurable number generally between 2-5)offers are selected for administration (along with the contractedoffer). Selection may be performed in a number of ways. For example, thetop N offers, by forecast, may be first selected, and then ranked byscore. From these the top ranked offers, the final selection of offersmay be made. In contrast, in some other embodiments, offers with a scoreabove a set threshold are first selected, and these may be ranked byforecast. From this forecast ranking the final offer selection may bemade. In yet another embodiment, a weighted composite metric for eachoffer may be generated. The composite score may be a set weight timesthe score, plus a normalized value of the forecast times a secondconfigured weight. The offers with the top composite metric are thenselected for deployment.

Forecasts are generated as a base forecast for the retail pricing of theproduct, plus an expected lift for the offer promotion. Base forecastsare collected from historical transaction log (t-log) data. Lift is ameasure of elasticity collected from cross store experimentation timesthe changed price of the promotion. Baseline t-log data is generallycollected from the prior two years, with seasonality, stock, store andgrowth trends adjusted for in an accounting model. Elasticities fromexperimentation are calculated and refined over time. The elasticitiesare utilized in conjunction with either current price or optimal price(depending on the type of forecast desired) to give an additive orsubtractive effect to the overall forecast.

The calculation of elasticities is generated by collecting daily feedsof transactions from retailers. These transaction logs are analyzed andadjusted by seasonality, out of stock events, and store specificadjustments. The data is further determined to be relevant based uponindications if the stores were compliant with a given offer. It iscommon for a store to adjust an offer unilaterally, or even discardingthe offer entirely, in some cases. This generally occurs when a storemanager believes she “knows better” than the system, there areinsufficient resources to update signage and inventory pricing software,or merely due to miscommunications between the offer generation systemand the retailer implementing the promotion. Data is further adjusted toaccount for out-of-stock events, and normalized across stores to accountfor different store attributes that may alter the normal velocity ofsales for the given product. Lastly, lifts and standard deviations aremodeled, and these are used to determine the top performing offers, andconfidence levels that they are “better” than the other offers.

Turning to FIG. 11, a flow chart of an example method for contract basedoffer generation is provided, as shown at 1100. Initially, one or modecontracts are received (at 1110) for offers of given products. Thecontracts are parsed (at 1120) to determine the important elements forthe contract. Contract parsing is provided in greater detail in relationto FIG. 12. Here the contracts are cross referenced for similarities inproducts that are subject to offers (at 1210). If there are multiplecontracts that apply to a single product (at 1220) the contract with thelonger duration is selected for testing (at 1230). This is becauselonger test periods tend to minimize isolated events that may skewtesting results. Once the contract that is applicable is selected, theimportant information in the contract is extracted. This includesidentifying product(s) implicated by the contract (at 1240), the offertype/structure and values of the offer (at 1250), as well as the datesof the offer (at 1260) including the promotional start date andduration. Additional information that may be extracted may includeContract ID, Promotion Type, SKU, Promo Quantity, Promo Price, Number ofStores, Description, Customer Name, Manufacturer ID, Manufacturer Name,Merchant ID, Merchant Name, Deal ID, Deal Type, Deal Calculation Type,Deal Amount, Base Price, Base Cost. This information from the contractoffer may be extracted by the system automatically, via humanintervention, or by a hybrid approach, as previously discussed ingreater detail. The final contract details are output for later usage(at 1270).

Returning to FIG. 11, the next step is to generate test offers (at 1130)that are to be tested alongside the contracted offers. FIG. 13 providesgreater detail of this offer generation process. Again, initially it isassumed that a number of test promotions have already, and arecontinually, being run in the retailers. As discussed above, thesepromotion activities are recorded as transaction logs from theretailers, and are collected by the offer generation system (at 1310).The transaction logs are then conditioned, as previously discussed, totake into account store/product variations, out of stock events, andcompliance adjustments. This test data is leveraged to generatedforecasts for different test offers (not illustrated) as discussed inconsiderable detail previously, preferably leveraging reinforcedlearning models. These forecasts for the various offers are used topopulate an offer bank for later review and selection of particularoffers for testing.

Subsequently, for the present test, a number of the test offers to bedeployed along with the contracted offer is determined (at 1330). Asdiscussed, this number typically varies between 2-5 additional testpromotions (not including the contract offer). However, the number maydynamically vary based upon the number of retail locations involved inthe promotion test, sales volume for the product involved, contractlength, or by user configuration. Likewise, the objective of the offerscan be set (at 1340). Offer objective may include hitting a target (ormaximizing) margin, profit maximization, sales volume, revenuemaximization, or the like. The possible offers are generated (orselected from the prepopulated offer bank) using reinforced learningtechniques (at 1350) as previously discussed. As noted, offer selectionis an optimization problem for the given objective. The offers are thenscored against the contracted offer (at 1360). Examples of thealgorithms leveraged for offer scoring are provided in Appendix A, aspreviously noted. This Appendix is entirely for illustrative purposes,and any similar scoring algorithms may be employed. Generally, thefurther away from the contract offer, or from user's expectations, thelarger a penalty is applied to the given test offer. The “best” testoffers are then selected (at 1370) based upon their score, forecasts orby some hybrid approach, as previously covered in some detail. In someembodiments, Thompson sampling may be leveraged to select the testoffers. In some select embodiments, scoring of the test offers isn'tperformed at all, and the objective optimization alone is leveraged todecide which offers to utilize in the promotion test.

Returning to FIG. 11, after the selection of the contract based testoffers, they may be assigned to retail locations for administration (at1140). The offer administration allocates offers among retailerlocations and/or in time periods designed to maximize orthogonalitybetween the following variables: Store sales, Store stock-out rates,Number of relevant SKUs carried in each store, Temporal effects,Discount depth, Buy quantity, and Offer structure. The offer allocationmay be done by using reinforcement learning techniques to their fullest,e.g., by an optimization problem in which model predictions anduncertainty are computed for each offer for each store for each period,and offers are allocated to maximize customer objectives while reducingmodel uncertainty.

After the offers are thus administered, users in the store redeem theoffers (at 1150) which becomes reflective in the transaction logs of theretailers. This transaction log data is then leveraged to performfurther analysis and refine the machine learning models (at 1160). Asnoted before, the transaction logs are processed and adjusted to accountfor temporal effects, store variations, out of stock events, andcompliance issues. Outliers and stock-outs are detected at the same timeand rely on iterating through records for every product-storecombination two times. For Pass 1, the steps include: set startinginactivity threshold to 20 days (configurable), mark an item as beingout of stock if the inactivity threshold is exceeded, record the averagequantity per day when the item was in stock, use the Poissondistribution to get upper and lower bounds for outliers using theaverage quantity (in some embodiments the system may use 99.9%(configurable) for the threshold. This threshold is desired to be highto make sure it is detecting true outliers and not responses to pricechanges.), and pick a better day threshold for detecting stock-outs:1+(int) (−Math.log(0.05)/(average quantity)). The stock out threshold isalso configurable. For the Pass 2: mark an item as being out of stock ifthe new inactivity threshold is exceeded (note, it is not assumed thatan item remains out of stock until the next item is sold. Instead, thesystem designates that it came back into stock for the number of daysequal to the threshold before the next item is sold), and mark an itemas being an outlier if the quantity is outside of the bounds that wascalculated in the previous pass.

After processing the transaction log data in this manner, lifts andstandard deviations are calculated, and these form the basis of theforecasts and confidence intervals of the models.

Moving forward, FIGS. 14-19 provide illustrations of example dashboardsfor the disclosed systems. It should be noted that these figures areexemplary only, and are not intended to limit the scope of the givensystems. In FIG. 14 categories of products are illustrated at 1400. Thecategories are for promotions that are in development or are beingadministered. The business objective for the given promotions areprovided, including the number of promotion events under the givenproduct category. Attributes being tested are listed, as is the datascientist or managing individual associated with the promotions.

When a product category is selected, the dashboard goes to a listing ofthe given promotion events that are associated with the given productcategory, as is seen in relation to FIG. 15 at 1500. Here is individualpromotions are listed, along with their statuses, the objective of thegiven promotion, the contract ID the promotion is associated with, startand end dates, and when applicable a listing of the top offer associatedwith the promotion event. Attribute tags associated with the promotionare also listed, along with the base price range (or singe number)associated with the product.

When a specific promotion is selected, the system progresses to adashboard for the given offer, as seen at FIG. 16 at 1600. Here thegiven offers that have been selected for the promotion (including thecontrol/contract offer) are listed. Each offer includes a rational forit being included, and the percent discount associated with the offer.As noted before, scoring of the offers tends to place a penalty foroffers that deviate from the contracted offer. As such, these discountpercentages are often the same as the control contract offer, or verysimilar, as seen here.

In the promotion screen, the user may also wish to view the execution ofthe promotion, as seen in relation to FIG. 17 at 1700. Here the storesare listed, along with their physical locations, if they are test orcontrol stores, and the offer ID that is deployed in the given store.The percentage of stores that the offers are each deployed at are alsoseen in the bar graph. As noted, deployment is selected to orthogonallyaccount for a number of variables, as previously touched upon.

Any given promotion event has the ability to be edited (at least beforeits deployment). An illustration for this offer editing dashboard isprovided at FIG. 18 at 1800. Here the control/contracted offer is seen,which cannot be edited. However, additional offers are presented basedupon how suited they are for deployment. Offers that scored the highestare listed at the top, and less attractive offers are presented indescending order. For each offer, the type, the rationale for the offer,and discount percentage are all presented. A user is able to manuallyoverride the automated offer selection in this editing dashboard, andselect different, more or fewer offers for testing.

In FIG. 19, the offer bank dashboard for all offers is presented, at1900. The category, product, discount, offer type, driver for the offer,and projected sales, unit volumes and margin are all presented for thegiven offers. As noted, forecasts for each offer are based upontransaction logs previously collected that have been adjusted andprocessed for lifts and standard deviations based upon the variablesassociated with the offers using reinforced learning techniques.

IV. System Embodiments

Now that the systems and methods for the generation and administrationof contract based offer testing have been provided, attention shall nowbe focused upon apparatuses capable of executing the above functions inreal-time. To facilitate this discussion, FIGS. 20A and 20B illustrate aComputer System 2000, which is suitable for implementing embodiments ofthe present invention. FIG. 20A shows one possible physical form of theComputer System 2000. Of course, the Computer System 2000 may have manyphysical forms ranging from a printed circuit board, an integratedcircuit, and a small handheld device up to a huge super computer.Computer system 2000 may include a Monitor 2002, a Display 2004, aHousing 2006, a server blades including one or more storage Drives 2008,a Keyboard 2010, and a Mouse 2012. Medium 2014 is a computer-readablemedium used to transfer data to and from Computer System 2000.

FIG. 20B is an example of a block diagram for Computer System 2000.Attached to System Bus 2020 are a wide variety of subsystems.Processor(s) 2022 (also referred to as central processing units, orCPUs) are coupled to storage devices, including Memory 2024. Memory 2024includes random access memory (RAM) and read-only memory (ROM). As iswell known in the art, ROM acts to transfer data and instructionsuni-directionally to the CPU and RAM is used typically to transfer dataand instructions in a bi-directional manner. Both of these types ofmemories may include any suitable of the computer-readable mediadescribed below. A Fixed Medium 2026 may also be coupledbi-directionally to the Processor 2022; it provides additional datastorage capacity and may also include any of the computer-readable mediadescribed below. Fixed Medium 2026 may be used to store programs, data,and the like and is typically a secondary storage medium (such as a harddisk) that is slower than primary storage. It will be appreciated thatthe information retained within Fixed Medium 2026 may, in appropriatecases, be incorporated in standard fashion as virtual memory in Memory2024. Removable Medium 2014 may take the form of any of thecomputer-readable media described below.

Processor 2022 is also coupled to a variety of input/output devices,such as Display 2004, Keyboard 2010, Mouse 2012 and Speakers 2030. Ingeneral, an input/output device may be any of: video displays, trackballs, mice, keyboards, microphones, touch-sensitive displays,transducer card readers, magnetic or paper tape readers, tablets,styluses, voice or handwriting recognizers, biometrics readers, motionsensors, brain wave readers, or other computers. Processor 2022optionally may be coupled to another computer or telecommunicationsnetwork using Network Interface 2040. With such a Network Interface2040, it is contemplated that the Processor 2022 might receiveinformation from the network, or might output information to the networkin the course of performing the above-described promotion offergeneration and redemption. Furthermore, method embodiments of thepresent invention may execute solely upon Processor 2022 or may executeover a network such as the Internet in conjunction with a remote CPUthat shares a portion of the processing.

Software is typically stored in the non-volatile memory and/or the driveunit. Indeed, for large programs, it may not even be possible to storethe entire program in the memory. Nevertheless, it should be understoodthat for software to run, if necessary, it is moved to a computerreadable location appropriate for processing, and for illustrativepurposes, that location is referred to as the memory in this disclosure.Even when software is moved to the memory for execution, the processorwill typically make use of hardware registers to store values associatedwith the software, and local cache that, ideally, serves to speed upexecution. As used herein, a software program is assumed to be stored atany known or convenient location (from non-volatile storage to hardwareregisters) when the software program is referred to as “implemented in acomputer-readable medium.” A processor is considered to be “configuredto execute a program” when at least one value associated with theprogram is stored in a register readable by the processor.

In operation, the computer system 2000 can be controlled by operatingsystem software that includes a file management system, such as a mediumoperating system. One example of operating system software withassociated file management system software is the family of operatingsystems known as Windows® from Microsoft Corporation of Redmond,Washington, and their associated file management systems. Anotherexample of operating system software with its associated file managementsystem software is the Linux operating system and its associated filemanagement system. The file management system is typically stored in thenon-volatile memory and/or drive unit and causes the processor toexecute the various acts required by the operating system to input andoutput data and to store data in the memory, including storing files onthe non-volatile memory and/or drive unit.

Some portions of the detailed description may be presented in terms ofalgorithms and symbolic representations of operations on data bitswithin a computer memory. These algorithmic descriptions andrepresentations are the means used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. An algorithm is, here and generally,conceived to be a self-consistent sequence of operations leading to adesired result. The operations are those requiring physicalmanipulations of physical quantities. Usually, though not necessarily,these quantities take the form of electrical or magnetic signals capableof being stored, transferred, combined, compared, and otherwisemanipulated. It has proven convenient at times, principally for reasonsof common usage, to refer to these signals as bits, values, elements,symbols, characters, terms, numbers, or the like.

The algorithms and displays presented herein are not inherently relatedto any particular computer or other apparatus. Various general purposesystems may be used with programs in accordance with the teachingsherein, or it may prove convenient to construct more specializedapparatus to perform the methods of some embodiments. The requiredstructure for a variety of these systems will appear from thedescription below. In addition, the techniques are not described withreference to any particular programming language, and variousembodiments may, thus, be implemented using a variety of programminglanguages.

In alternative embodiments, the machine operates as a standalone deviceor may be connected (e.g., networked) to other machines. In a networkeddeployment, the machine may operate in the capacity of a server or aclient machine in a client-server network environment or as a peermachine in a peer-to-peer (or distributed) network environment.

The machine may be a server computer, a client computer, a personalcomputer (PC), a tablet PC, a laptop computer, a set-top box (STB), apersonal digital assistant (PDA), a cellular telephone, an iPhone, aBlackberry, a processor, a telephone, a web appliance, a network router,switch or bridge, or any machine capable of executing a set ofinstructions (sequential or otherwise) that specify actions to be takenby that machine.

While the machine-readable medium or machine-readable storage medium isshown in an exemplary embodiment to be a single medium, the term“machine-readable medium” and “machine-readable storage medium” shouldbe taken to include a single medium or multiple media (e.g., acentralized or distributed database, and/or associated caches andservers) that store the one or more sets of instructions. The term“machine-readable medium” and “machine-readable storage medium” shallalso be taken to include any medium that is capable of storing, encodingor carrying a set of instructions for execution by the machine and thatcause the machine to perform any one or more of the methodologies of thepresently disclosed technique and innovation.

In general, the routines executed to implement the embodiments of thedisclosure may be implemented as part of an operating system or aspecific application, component, program, object, module or sequence ofinstructions referred to as “computer programs.” The computer programstypically comprise one or more instructions set at various times invarious memory and storage devices in a computer, and when read andexecuted by one or more processing units or processors in a computer,cause the computer to perform operations to execute elements involvingthe various aspects of the disclosure.

Moreover, while embodiments have been described in the context of fullyfunctioning computers and computer systems, those skilled in the artwill appreciate that the various embodiments are capable of beingdistributed as a program product in a variety of forms, and that thedisclosure applies equally regardless of the particular type of machineor computer-readable media used to actually effect the distribution

While this invention has been described in terms of several embodiments,there are alterations, modifications, permutations, and substituteequivalents, which fall within the scope of this invention. Althoughsub-section titles have been provided to aid in the description of theinvention, these titles are merely illustrative and are not intended tolimit the scope of the present invention. It should also be noted thatthere are many alternative ways of implementing the methods andapparatuses of the present invention. It is therefore intended that thefollowing appended claims be interpreted as including all suchalterations, modifications, permutations, and substitute equivalents asfall within the true spirit and scope of the present invention.

What is claimed is:
 1. A computer implemented method for generatingcontract based offers comprising: receiving a contract for a promotionaloffer on a product; extracting data from the contract; accessing anoffer bank; and selecting a plurality of test offers from the offer bankby scoring each offer in the offer bank against the extracted data. 2.The method of claim 1, further comprising deploying the promotionaloffer and the selected plurality of test offers in a plurality of retaillocations.
 3. The method of claim 2, wherein the deploying is performedto maximize orthogonality between the following variables: store sales,store out of stock rates, number of relevant SKUs carried in each store,temporal effects, discount depth, buy quantity and offer structure. 4.The method of claim 1, further comprising selecting a number of testoffers to run in-market using reinforcement learning techniques.
 5. Themethod of claim 4, wherein Thompson sampling is used to select thenumber.
 6. The method of claim 1, wherein the offer bank is populatedwith forecasted offers.
 7. The method of claim 6, wherein the forecastsare based upon transaction logs of a plurality of retailers.
 8. Themethod of claim 7, wherein the transaction logs are adjusted forcompliance by the given retailer, estimated out of stock events,normalized across stores to account for different store attributes, andadjusted for temporal effects.
 9. The method of claim 8, furthercomprising applying machine learning to the adjusted transaction logs todetermine lift and standard deviation for a given test offer.
 10. Themethod of claim 9, wherein the forecasts are a baseline function of timefrom the transaction log data plus elasticity from cross storeexperiments times a change in price, where in the elasticity iscalculated as a function of the lift, and a confidence for the forecastis calculated as a function of the standard deviation.
 11. A computerproduct comprising non-transitory computer readable medium, which whenexecuted on a computer system causes the computer system to perform thesteps of: receiving a contract for a promotional offer on a product;extracting data from the contract; accessing an offer bank; andselecting a plurality of test offers from the offer bank by scoring eachoffer in the offer bank against the extracted data.
 12. The computerproduct of claim 11, further comprising deploying the promotional offerand the selected plurality of test offers in a plurality of retaillocations.
 13. The computer product of claim 12, wherein the deployingis performed to maximize orthogonality between the following variables:store sales, store out of stock rates, number of relevant SKUs carriedin each store, temporal effects, discount depth, buy quantity and offerstructure.
 14. The computer product of claim 11, wherein when thecomputer readable product when executed further performs the step ofselecting a number of test offers to run in-market using reinforcementlearning techniques.
 15. The computer product of claim 14, whereinThompson sampling is used to select the number.
 16. The computer productof claim 11, wherein the offer bank is populated with forecasted offers.17. The computer product of claim 16, wherein the forecasts are basedupon transaction logs of a plurality of retailers.
 18. The computerproduct of claim 17, wherein the transaction logs are adjusted forcompliance by the given retailer, estimated out of stock events,normalized across stores to account for different store attributes, andadjusted for temporal effects.
 19. The computer product of claim 18,wherein when the computer readable product when executed furtherperforms the step of applying machine learning to the adjustedtransaction logs to determine lift and standard deviation for a giventest offer.
 20. The computer product of claim 19, wherein the forecastsare a baseline function of time from the transaction log data pluselasticity from cross store experiments times a change in price, wherein the elasticity is calculated as a function of the lift, and aconfidence for the forecast is calculated as a function of the standarddeviation.